The present invention relates to data transmission in tightly coupled point-to-point system architectures and, more specifically, to credit based links in such architectures.
Point-to-point architectures are fast gaining acceptance as higher bandwidth alternatives to traditional bus architectures in tightly coupled systems. In such systems, individual devices or nodes may be interconnected in a variety of topologies using single or pairs of unidirectional point-to-point links between the devices. An example of such a system is a multiprocessor computing system in which processing nodes communicate with each other and system memory via such an architecture. One such point-to-point architecture is the HyperTransport™ architecture pioneered by AMD of Sunnyvale, Calif.
In some point-to-point systems, the transmission of data between two devices over a particular point-to-point link may be facilitated using a credit based approach. According to such an approach, data credits are provided to the transmitting device indicating that the receiving device is ready to receive some unit of data, e.g., a packet. The number of data credits typically corresponds to the amount of buffer space available in the receiving device to store data for consumption by the receiving device. When the transmitting device is ready to transmit data, it determines whether it has any data credits and, if so, decrements a credit counter and transmits the data to the receiving device. The receiving device typically releases the data credit back to the transmitting device (which then increments its credit counter) when the transmitted data is consumed from the buffer.
The latency associated with the transmission of a particular data packet and the subsequent return of the corresponding data credit may be understood with reference to the diagram of FIG. 1. The diagram illustrates an exemplary latency associated with a receiving device RX A consuming a 64-byte data packet and then returning the corresponding data credit to a transmitting device TX B. Assuming the numbers shown for transmitter, receiver and internal device latencies results in a total loop delay of approximately 324 ns. Further assume that the link between the devices is a 3.2 GB/second link. In order to “hide” the loop latency, i.e., allow the link to operate at full bandwidth without stalling, sufficient buffer space to store 1036 (324×3.2) bytes of data must be provided in the receiving device RX A, e.g., seventeen 64-byte buffers.
Devices in point-to-point architectures may employ a data transmission protocol which transmits data in different “virtual” channels. That is, at the protocol level, data packets are segregated and handled differently for a variety of purposes. In general, virtual channels are set up such that none of the virtual channels is allowed to stall as a direct result of another virtual channel stalling. This is accomplished by having dedicated buffers for each virtual channel. That is, each buffer and its corresponding data credit is dedicated for the transmission and storage of data packets in a particular virtual channel. And because the receiving device cannot predict the distribution of data traffic over the different virtual channels, if full bandwidth operation is to be supported for every virtual channel, the data credits and corresponding buffer space required for full bandwidth operation must be replicated for each virtual channel. This accounts for the condition in which all of the data are being transmitted in a particular virtual channel. So, for example, if the system of FIG. 1 had three virtual channels, full bandwidth operation would require 51 buffers rather than 17.
In the example of FIG. 1, each data credit requires a 512 bit buffer array in the receiving device, a significant expense in chip area. In addition, larger arrays tend to have longer access times. This forces the designer to make design choices that may limit performance on some virtual channels in order to fit the data buffer in either the chip area or the timing constraints. It is therefore desirable to provide techniques which mitigate or avoid the negative consequences associated with such design choices.